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AMENDMENTS TO THE CLAIMS 

1. (Cancelled) 

2. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
passback direction is defined based on at least one of an image of said video monitored area and 
video of said video monitored area. 

3. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
passback direction is user defined. 

4. (Original) A computer-readable medium as in claim 3, wherein said passback direction is 
determined via a graphical user interface to identify said passback direction in at least one of an 
image of said video monitored area and video of said video monitored area. 

5. (Original) A computer-readable medium as in claim 4, wherein said graphical user 
interface is adapted to permit a user to draw said passback direction on at least one of an image of 
said video monitored area and video of said video monitored area. 

6. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
defining said passback direction comprises learning said passback direction through observation of 
said video monitored area. 

7. (Original) A computer-readable medium as in claim 6, wherein said learning said 
passback direction comprises: 

learning a normal direction for said video monitored area based on said observation of said 
video monitored area; and 

determining said passback direction based on said normal direction. 
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8. (Original) A computer-readable medium as in claim 6, wherein said passback direction is 
learned as a function of time. 

9. (Original) A computer-readable medium as in claim 8, wherein said passback direction is 
learned for a first time period, and wherein said defining said passback direction further comprises 
learning at least one additional passback direction for said video monitored area for at least one 
additional time period based on said observation of said video monitored area. 

10. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
passback direction is defined in association with at least one of a region of interest of said video 
monitored area, a time interval of detection, and a sensitivity of detection. 

11. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
accessing video comprises accessing video in real time from a video camera. 

12. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
accessing video comprises accessing stored video. 

13. (Cancelled) 

14. (Previously Presented) A computer-readable medium, comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 

extracting foreground from said collected video to obtain extracted 
foreground; 

detecting trackable features based on said extracted foreground; 
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tracking said trackable features based on said extracted foreground to obtain 
extracted tracks; and 

filtering said extracted tracks; 
detecting passback events based on the passback direction and said extracted tracks; and 
initiating an action based on the detected passback events. 

15. (Original) A computer-readable medium as in claim 14, wherein said extracting 
foreground is based on pixel statistics of a current frame and at least one past frame. 

16. (Original) A computer-readable medium as in claim 14, wherein said extracting 
foreground is based on three-frame motion differencing. 

17. (Original) A computer-readable medium as in claim 14, wherein said detecting trackable 
features comprises: 

subdividing foreground of said collected video into cells; 
determining if each cell is appropriate for tracking; and 
designating appropriate cells for tracking as trackable features. 

18. (Previously Presented) A computer-readable medium as in claim 17, wherein said 
determining if each cell is appropriate for tracking is based on at least one of texture in each cell, 
intensity range in each cell, number of different intensity values in each cell, presence of at least one 
edge in each cell, or presence of at least one corner in each cell. 

19. (Original) A computer-readable medium as in claim 14, wherein said tracking said 
trackable features comprises: 

correlating each trackable feature with previous tracked features to obtain correlated 
trackable features and correlated previous tracked features; 

comparing directions of said correlated trackable features and said correlated previous 
tracked features; 
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designating said correlated trackable features as tracked features if said directions are 
consistent; and 

updating said correlated previous tracked features with said correlated trackable features if 
said directions are consistent; and 

wherein said filtering said extracted tracks comprises filtering said tracked features. 

20. (Original) A computer-readable medium as in claim 19, wherein said correlating each 
trackable feature comprises performing two-dimensional correlation in an area predicted from said 
previous tracked features to contain said trackable feature. 

21. (Original) A computer-readable medium as in claim 19, wherein said correlating each 
trackable feature comprises performing one-dimensional correlation on a horizontal projection and a 
vertical projection of an area predicted from said previous tracked features to contain said trackable 
feature. 

22. (Previously Presented) A computer-readable medium as in claim 19, wherein said 
updating said correlated previous tracked features is based on at least one of a direction or an age of 
said correlated trackable features. 

23. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
filtering said extracted tracks comprises at least one of filtering noise from said extracted tracks, 
filtering out nuisance extracted tracks from said extracted tracks, filtering out extracted tracks from 
said extracted tracks that are not sufficiently long lived, or filtering out extracted tracks from said 
extracted tracks that are inconsistent with other nearby extracted tracks. 

24. (Previously Presented) A computer-readable medium, comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of: 

defining a passback direction for a video monitored area; 
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accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 

extracting foreground from said collected video to obtain extracted 
foreground; 

determining optical flow based on said extracted foreground to obtain said 
extracted tracks; and 

filtering said extracted tracks; 
detecting passback events based on the passback direction and the extracted tracks; and 
initiating an action based on the detected passback events. 

25. (Original) A computer-readable medium as in claim 24, wherein said extracting 
foreground is based on pixel statistics of a current frame and at least one past frame. 

26. (Original) A computer-readable medium as in claim 24, wherein said extracting 
foreground is based on three-frame motion differencing. 

27. (Original) A computer-readable medium as in claim 24, wherein said determining 
optical flow comprises: 

determining current flow vectors for foreground pixels of said extracted foreground; 
comparing current and previous flow vectors to obtain consistent current flow vectors; and 
aggregating consistent current flow vectors into cumulative flow vectors. 

28. (Original) A computer-readable medium as in claim 24, wherein said filtering said 
extracted tracks comprises at least one of filtering noise from said extracted tracks, filtering out 
nuisance extracted tracks from said extracted tracks, filtering out extracted tracks from said 
extracted tracks that are not sufficiently long lived, and filtering out extracted tracks from said 
extracted tracks that are inconsistent with other nearby extracted tracks. 

29. (Cancelled) 
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30. (Previously Presented) A computer-readable medium, comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 

comparing a track determined from the collected video to the passback direction to detect 
whether any passback event occurred, comprising: 

representing said track as a track vector; 

representing said passback direction as a passback direction vector; 
determining a dot product of said track vector and said passback direction 

vector; 

comparing said dot product to a threshold to determine whether said track 
vector is substantially aligned with said passback direction; and 

when said track vector is substantially aligned with said passback direction, 
filtering said track vector by at least one of spatial filtering or temporal filtering to 
determine whether a sufficient number of track vectors are substantially aligned with 
said passback direction; 

detecting passback events based on the determination; and 
initiating an action based on the detected passback events. 

31. (Cancelled) 

32. (Previously Presented) A computer-readable medium, comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
analyzing the collected video; 



8 



Application No. 10/766,949 

Reply to Office Action of December 1, 2008 



Docket No.: 371 12-192025 



detecting passback events in the analyzed video based on the passback direction; and 
generating a report for each detected passback event, wherein said report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction of 
each detected passback event, a location of each detected passback event, a size of a crossing area of 
each detected passback event, an indication of occlusion of each detected passback event, at least 
one snapshot of each detected passback event, at least one video of each detected passback event, a 
number of detected passback events over a time interval, or a time histogram of detected passback 
events over a time interval. 

33. (Previously Presented) A computer-readable medium as in claim 14, wherein initiating 
said action comprises initiating a triggered response for each detected passback event. 

34. (Previously Presented) A computer-readable medium as in claim 33, wherein said 
triggered response comprises at least one of initiating an alarm, controlling an audible alarm system, 
controlling a silent alarm system, accessing an alerting device, accessing an alerting system, sending 
an alert, logging alert data to a database, taking a snapshot of each detected passback event, culling 
a snapshot of each detected passback event from said video, recording video of each detected 
passback event, controlling a camera to zoom in on each detected passback event, controlling a 
camera to track each detected passback event, performing recognition of each detected passback 
event, closing at least one door, or controlling an access control system to at least one of lock, 
unlock, open, and close at least one portal. 

35. (Previously Presented) A computer-readable medium as in claim 14, further comprising 
defining at least one of an option for reporting any detected passback event or an option for 
responding to any detected passback event. 

36. (Previously Presented) A computer-readable medium as in claim 14, wherein said 
method is adapted to detect passback events in real time. 
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37. (Previously Presented) A method to detect passback events comprising: 

defining a passback direction for a video monitored area; 

accessing video collected from said video monitored area; 

extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted foreground, 

detecting trackable features based on the extracted foreground, 

tracking the trackable features based on the extracted foreground to obtain 

extracted tracks, and 

filtering the extracted tracks; 

detecting passback events in said video monitored area based on said passback direction and 
extracted tracks; and 

initiating an action based on the detected passback events. 



38. (Previously Presented) A system to detect passback events, comprising: 
at least one video camera to monitor at least one video monitored area and to obtain video of 
said video monitored area; 

an analysis system coupled to said video camera, said analysis system comprising a 
computer system and a computer-readable medium, said computer-readable medium comprising 
software to control said computer system according to a method, said method comprising: 

extracting foreground from the collected video to obtain extracted 
foreground, detecting trackable features based on the extracted foreground, 

tracking the trackable features based on the extracted foreground to obtain 
extracted tracks, and 

filtering the extracted tracks to detect passback events in said video 
monitored area based on a passback direction and the extracted tracks for said video 
monitored area; and 

a user interface coupled to said analysis system. 
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39. (Previously Presented) An apparatus to detect passback events adapted to perform 
operations comprising a method of: 

defining a passback direction for a video monitored area; 

accessing video collected from said video monitored area; 

extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted foreground, 

detecting trackable features based on the extracted foreground, 

tracking the trackable features based on the extracted foreground to obtain 

extracted tracks, and 

filtering the extracted tracks; 

detecting passback events in said video monitored area based on said passback direction and 
extracted tracks; and 

initiating an action based on the detected passback events. 

40. (Previously Presented) The apparatus of claim 39, wherein said apparatus comprises 
application-specific hardware to emulate at least one of a computer or software adapted to perform 
said operations. 

41. (Previously Presented) A method to detect passback events, comprising: 
defining a passback direction for a video monitored area; 

accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted 
foreground, 

determining optical flow based on the extracted foreground to obtain the 
extracted tracks, and 

filtering the extracted tracks; 
detecting passback events based on the passback direction and the extracted tracks; and 
initiating an action based on the detected passback events. 
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42. (Previously Presented) A method to detect passback events, comprising: 
defining a passback direction for a video monitored area; 

accessing video collected from the video monitored area; 

comparing a track determined from the collected video to the passback direction to detect 
whether any passback event occurred, comprising: 

representing the track as a track vector, 

representing the passback direction as a passback direction vector, 
determining a dot product of the track vector and the passback direction 

vector, 

comparing the dot product to a threshold to determine whether the track 
vector is substantially aligned with the passback direction, and 

when the track vector is substantially aligned with the passback direction, 
filtering the track vector by at least one of spatial filtering or temporal filtering to 
determine whether a sufficient number of track vectors are substantially aligned with 
the passback direction; 

detecting passback events based on the determination; and 
initiating an action based on the detected passback events. 

43. (Previously Presented) A method to detect passback events, comprising: 
defining a passback direction for a video monitored area; 

accessing video collected from the video monitored area; 
analyzing the collected video; 

detecting passback events in the analyzed video based on the passback direction; and 
generating a report for each detected passback event, wherein the report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction of 
each detected passback event, a location of each detected passback event, a size of a crossing area of 
each detected passback event, an indication of occlusion of each detected passback event, at least 
one snapshot of each detected passback event, at least one video of each detected passback event, a 
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number of detected passback events over a time interval, or a time histogram of detected passback 
events over a time interval. 

44. (Previously Presented) A system to detect passback events, comprising: 

a video camera to monitor a video monitored area and obtain video of the video monitored 

area; 

an analysis system coupled to the video camera, the analysis system comprising a computer 
system and a computer-readable medium, the computer-readable medium comprising software to 
control the computer system according to a method, the method comprising: 

extracting foreground from the obtained video to obtain extracted foreground, 
determining optical flow based on the extracted foreground to obtain 
extracted tracks, and 

filtering the extracted tracks to detect passback events in the video monitored 
area based on a passback direction and the extracted tracks; 

and 

a user interface coupled to the analysis system. 

45. (Previously Presented) A system to detect passback events, comprising: 

a video camera to monitor a video monitored area and obtain video of the video monitored 

area; 

an analysis system coupled to the video camera, the analysis system comprising a computer 
system and a computer-readable medium, the computer-readable medium comprising software to 
control the computer system according to a method, the method comprising: 

determining a track in the obtained video, 

representing the track as a track vector, 

representing a passback direction as a passback direction vector, 
determining a dot product of the track vector and the passback direction 

vector, 
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comparing the dot product to a threshold to determine whether the track 
vector is substantially aligned with the passback direction, 

when the track vector is substantially aligned with the passback direction, 
filtering the track vector by at least one of spatial filtering or temporal filtering to 
determine whether a substantial number of track vectors are substantially aligned 
with the passback direction, and 

detecting the passback events in the video monitored area based on the 
number of track vectors which are substantially aligned with the passback direction; 

and 

a user interface coupled to the analysis system. 

46. (Previously Presented) A system to detect passback events, comprising: 

a video camera to monitor a video monitored area and obtain video of the video monitored 

area; 

an analysis system coupled to the video camera, the analysis system comprising a computer 
system and a computer-readable medium, the computer-readable medium comprising software to 
control the computer system according to a method, the method comprising: 

analyzing the obtained video to detect passback events in the video monitored 

area based on a passback direction for the video monitored area, and 
generating a report for each detected passback event, 
wherein the report comprises at least one of a time of each detected passback event, a date of each 
detected passback event, a direction of each detected passback event, a location of each detected 
passback event, a size of a crossing area of each detected passback event, an indication of occlusion 
of each detected passback event, at least one snapshot of each detected passback event, at least one 
video of each detected passback event, a number of detected passback events over a time interval, or 
a time histogram of detected passback events over a time interval; and 

a user interface coupled to the analysis system. 
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47. (Previously Presented) An apparatus to detect passback events adapted to perform 
operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted 
foreground, 

determining optical flow based on the extracted foreground to obtain the 
extracted tracks, and 

filtering the extracted tracks; 
detecting passback events based on the passback direction and the extracted tracks; and 
initiating an action based on the detected passback events. 

48. (Previously Presented) An apparatus to detect passback events adapted to perform 
operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 

comparing a track determined from the collected video to the passback direction to detect 
whether any passback event occurred, comprising: 

representing the track as a track vector, 

representing the passback direction as a passback direction vector, 
determining a dot product of the track vector and the passback direction 

vector, 

comparing the dot product to a threshold to determine whether the track 

vector is substantially aligned with the passback direction, and 

when the track vector is substantially aligned with the passback direction, 
filtering the track vector by at least one of spatial filtering or temporal filtering 

to determine whether a sufficient number of track vectors are substantially aligned 

with the passback direction; 
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detecting passback events based on the determination; and 
initiating an action based on the detected passback events. 

49. (Previously Presented) An apparatus to detect passback events adapted to perform 
operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
analyzing the collected video; 

detecting passback events in the analyzed video based on the passback direction; and 
generating a report for each detected passback event, wherein the report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction of 
each detected passback event, a location of each detected passback event, a size of a crossing area of 
each detected passback event, an indication of occlusion of each detected passback event, at least 
one snapshot of each detected passback event, at least one video of each detected passback event, a 
number of detected passback events over a time interval, or a time histogram of detected passback 
events over a time interval. 
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